Address high cpu usage with AWSCRTHTTPClient#573
Conversation
jonathan343
force-pushed
the
high-cpu-fix
branch
from
b24f494 to
b68df1b
Compare
| [project.optional-dependencies] | ||
| awscrt = [ | ||
| "awscrt>=0.23.10", | ||
| "awscrt~=0.28.2", |
There was a problem hiding this comment.
Note
Setting awscrt v0.28.2 is necessary since we need the change added in awslabs/aws-crt-python#688. Also, this was previously unbounded which has been fine for now, but constraining it to ~=0.28.2 is much safer for customers
| ): | ||
| # AsyncByteStream has async read method but is not iterable | ||
| while True: | ||
| chunk = await body.read(65536) # Read in 64KB chunks |
There was a problem hiding this comment.
Why this number in particular?
There was a problem hiding this comment.
It was a common buffer size I'd come across, however, I do see our default value is -1 which will read all available data at once.
Do you prefer going with the default?
There was a problem hiding this comment.
Updated to use the default in 02e3e20. Let me know what you think
There was a problem hiding this comment.
The default is for the protocol - when you call .read on a file-like object, for instance, it just reads everything. We should read in chunks, i was just wondering why that size in particular. Just make it a constant or something that's configurable on the crt client and we'll be good.
JordonPhillips
left a comment
JordonPhillips
left a comment
There was a problem hiding this comment.
See response to other comment - we should still limit how much we read at a time from a file-like object, I just wanted to know why that number.
2 participants
Summary
This PR integrates with the new aync CRT APIs introduced in awslabs/aws-crt-python#658 with the goal of addressing the high CPU usage reported in aws/aws-sdk-python#11.
The changes in this PR have been tested and successfully reduce CPU usage for examples such as the bidirectional nova_sonic.py. This change will be available in a new versions of
smithy-httpandaws-sdk-bedrock-runtimewith no changes to the user experience.Testing
gh pr checkout 573uv venv && source .venv/bin/activatemake installuv pip install aws-sdk-bedrock-runtime psutil pyaudio rxpython monitor_cpu.py bidirection.pyBy submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.